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ABSTRACT 


Education abroad programs are becoming more common as a mechanism for developing the 
global competencies of engineering graduates. An increasing body of research shows that intercul¬ 
tural learning does not occur de facto in such programs. This study used quantitative and qualitative 
methods to explore changes in students’ intercultural sensitivity over the course of a project-based 
education abroad experience in Thailand that was designed with proactive interventions to foster 
intercultural learning. In addition, we aimed to compare the intercultural development of students 
in U.S.-only project teams with those who were in cross-national teams with students from a part¬ 
ner university. Although the study group as a whole did not show a statistically significant gain in 
a quantitative measure of intercultural sensitivity, greater gains were seen from students in cross¬ 
national teams. A few qualitative results suggest learning that was not evident from quantitative 
measures. Overall, this study reinforces the challenges of facilitating and assessing intercultural 
learning during education abroad. 

Key words: Intercultural learning, global competency, study abroad, international education. 


INTRODUCTION 


Competency to work effectively in domestic and cross-national teams characterized by cultural 
diversity has been identified as increasingly important for professional engineering and scientific prac¬ 
tice and for citizenship in the 21st century (Downey et at, 2006; Duderstadt, 2008; Galloway, 2008; 
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Grandin & Hirleman, 2009; NAE, 2004; Parkinson, 2009). For example, the National Academy of 
Engineering (2004) has argued that US economic competitiveness increasingly requires engineers 
who can work across cultures. Downey et al. (2006, p. 119) also describe how different cultures en¬ 
gender fundamentally different ways of framing and approaching engineering problems, and call 
for “educational and work experiences with people who were raised and trained in other countries” 
as preparation for global competency and leadership. 

Education abroad programs have particular potential to promote intercultural competence, and 
a number of universities have set ambitious goals for providing engineering students with an inter¬ 
national experience (Parkinson, 2007). Historically, engineering undergraduates in the U.S. have not 
participated in education abroad as much as other college majors. According to the most recent 
statistics (2012-13), 4.7% of US bachelor’s degrees were awarded in engineering (National Center for 
Education Statistics, 2014) while 4.1% of study abroad participants were engineering students (Institute 
of International Education, 2014). Participation has increased in the past decade, however, in parallel 
with broader national trends. Between 2004 and 2014, the number of U.S. engineering undergraduates 
participating in study abroad increased by about 134% (Institute of International Education, 2015). 

Developing students’ intercultural competence or global competence is usually a stated or implicit 
goal of education abroad programs for engineering students, and program assessment efforts often 
include a quantitative measure of student development in this area. Numerous U.S. universities 
developing international programs for engineering students are using pre- and post-administration 
of the IDI as a program assessment measure (Arzberger et al., 2010; Bland, 2010; Georgia Insti¬ 
tute of Technology, 2011; Mayhew, Eljamal, Dey, & Pang, 2005; Paterson, 2010; Thompson, Jesiek, 
& Atkinson, 2010). The common use of the IDI likely is due to a number of factors: its grounding 
in a theoretical framework, best practices used in development of the instrument, demonstrated 
validity and reliability, and its immunity to social desirability bias (Paige, Jacobs-Cassuto, Yershova, 
& DeJaeghere, 2003). 

Many studies utilizing the IDI as a measure of intercultural competence suggest that intercultural 
learning during study abroad is more challenging to achieve than one might expect. Michael Vande 
Berg, a scholar of US study abroad, notes that in the traditional paradigm of US student learning 
abroad, it is assumed that participants naturally learn through experience, exposure, and immersion 
in another culture (Vande Berg, 2009). Recent studies have shown that assumption to be largely 
unsupported, however (Vande Berg, 2009; Vande Berg, Connor-Linton, & Paige, 2009). Many stu¬ 
dents in a variety of education abroad programs show negligible or very small gains in intercultural 
competence as measured by the IDI. 

Experts in education abroad argue for, and have evidence to support, a new paradigm of study 
abroad whereby educators design and deliver “proactive learning interventions” to create an 
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environment that balances challenge with support and engages students in active testing of and 
reflection on cross-cultural encounters (Vande Berg et at, 2009). This paper describes an effort to 
effect that paradigm change in an experiential education abroad program at Worcester Polytechnic 
Institute (WPI)— shifting from a somewhat laissez-faire approach to proactive interventions intended 
to foster intercultural learning among engineering and science undergraduates. 

This paper proceeds with a review of definitions of intercultural and global competence, ultimately 
focusing on the characteristic of intercultural sensitivity. We then introduce the Developmental 
Model of Intercultural Sensitivity, which is the theoretical framework of the IDI, and explain details 
of the instrument. Next we present a summary of findings from prior studies of education abroad 
programs that use the IDI as an outcomes measure, followed by a description of a particular educa¬ 
tion abroad program at WPI and the intervention we used to support development of intercultural 
sensitivity. 


LITERATURE REVIEW 


Defining, Framing, and Measuring Intercultural Competence 

Numerous scholars within and outside the engineering education community have attempted to 
define intercultural or global competence and its components (Deardorff, 2006; Grandin & Hedderich, 
2009; Hunter, White, & Godbey, 2006; Lohmann, Rollins, & Hoey, 2006; Parkinson, Harb, & Magleby, 
2009). The top-rated definition of intercultural competence in a Delphi study of intercultural scholars 
was “the ability to communicate effectively and appropriately in intercultural situations based on 
one’s intercultural knowledge, skills, and attitudes” (Deardorff, 2006, p. 247). Another Delphi study 
incorporating business representatives from transnational corporations defined global competence 
as “having an open mind while actively seeking to understand cultural norms and expectations of 
others, leveraging this gained knowledge to interact, communicate and work effectively outside one’s 
environment” (Hunter et at, 2006, p. 270). These studies also identify the constituent components 
of intercultural or global competence—typically a large set of attitudes, skills, behaviors, and knowl¬ 
edge. One component that is included in virtually all definitions centers on the theme of cultural 
differences: for example, identifying, understanding, and/or accepting cultural differences (Hunter 
et al., 2006; Lohmann et al., 2006; Parkinson et al., 2009); perceptual understanding—recognition 
that one’s worldview is not a universal perspective (Grandin & Hedderich, 2009); and understanding 
and avoiding ethnocentrism (Parkinson et al., 2009). Numerous authors argue that this ability to 
navigate cultural differences is an important professional skill for engineering graduates (Lohmann 
et al., 2006; Grandin & Hedderich, 2009; Parkinson et at, 2009). 
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The ability to navigate cultural differences is well-encompassed by the characteristic of intercultural 
sensitivity defined by Bhawuk and Brislin (1992, p. 414) as “sensitivity to the importance of cultural 
differences and to the points of view of people in other cultures.” Milton Bennett’s Developmental 
Model of Intercultural Sensitivity (DMIS) posits that as peoples’ experience of cultural difference 
becomes more complex, their potential competence in intercultural situations is enhanced (Bennett, 
1993). Bennett developed the DMIS using a grounded theory approach, drawing on more than 20 
years of empirical observations of how people develop intercultural competence. As such, the model 
is positioned within the epistemological and methodological framework of positivism, which assumes 
that experiences can be measured and analyzed scientifically and constructed into generalizable 
knowledge (Bernard, 1998). As a stage model of cognitive development, intercultural sensitivity is 
viewed as a continuum in developmental terms rather than as a static position. Furthermore, the DMIS 
suggests that experiencing an intercultural encounter requires construing the event and discriminat¬ 
ing cultural differences. In other words, it models development of worldview structure rather than 
skill acquisition or attitude change (Hammer & Bennett, 1998). Although the DMIS does not capture 
objective behavior during intercultural experiences, the model assumes that particular kinds of cog¬ 
nitive processing, attitudes, and behaviors are typically associated with each developmental stage. 

The first three stages of the DMIS (Denial, Defense, Minimization) are ethnocentric in nature, 
where one’s own cultural worldview tends to be latent but is the central reality in which other cul¬ 
tures are experienced. The other stages (Acceptance, Adaptation, Integration) are ethnorelative in 
nature, meaning that one’s own cultural worldview is understood and experienced as one among 
many that are possible and valid (Bennett, 1993). The ethnocentric stages represent various ways of 
avoiding cultural difference, while the ethnorelative stages involve various ways of seeking cultural 
difference (Hammer & Bennett, 1998). 

In order to measure the DMIS orientations for use in training and assessment, Hammer, Bennett, 
and Wiseman (2003) created the Intercultural Development Inventory (IDI) using scale construc¬ 
tion guidelines and best practices for development of cross-cultural instruments. The reliability, 
content validity, and construct validity of the IDI have been established (Hammer et al., 2003; Paige 
et al., 2003). Unlike several other instruments in this domain, the IDI shows no systematic effects 
by gender, age, or educational level. In addition, the instrument has been shown not to be subject 
to social desirability bias, meaning that participants’ responses appear not to be influenced by the 
natural tendency to provide socially desirable responses. This is noteworthy since tests of inter¬ 
cultural competence are often criticized on the assumption that the “correct” answers are obvious 
(Hammer & Bennett, 1998). 

The IDI is a 50-item paper-and-pencil or online questionnaire that asks respondents to rate each 
item on a 5-level scale of agreement (l=disagree, 2=disagree somewhat more than agree, 3=disagree 


4 


SPRING 2017 





ADVANCES IN ENGINEERING EDUCATION 

Influence of an Education Abroad Program on the Intercultural Sensitivity 
of STEM Undergraduates: A Mixed Methods Study 



Table 7. Scales Measured by the Intercultural Development Inventory (IDI). 


Scale 

Explanation 

Ethnocentrism 

Denial/Defense (DD) 

Simplifies and/or polarizes cultural difference 


Denial Cluster 

Disinterest in cultural difference or avoidance of interaction 
with cultural difference 


Defense Cluster 

Tendency to view the world in terms of “us and them ” where 
“us” is superior 


Defense-Reversal (R) 

Reverses “us” and “them” polarization, where “them” is superior 


Minimization (M) 

Highlights cultural commonality and universal values 


Similarity Cluster 

Tendency to assume that people from other cultures are 
basically “like us” 


Universalism Cluster 

Tendency to apply one's own cultural values to other cultures 

Ethnorelativism 

Acceptance/Adaptation (AA) 

Can comprehend and accommodate complex cultural difference 


Acceptance Cluster 

Tendency to recognize patterns of cultural difference in 
one’s own and other cultures 


Adaptation Cluster 

Tendency to shift cognitive frame and behavioral codes 
according to cultural context 


Integration/ Encapsulated Marginality (EM) 

Incorporates a multicultural identity with combined or 
confused cultural perspectives 



some and agree some, 4=agree somewhat more than disagree, and 5=agree). An example item is: 
People from our culture are less tolerant compared to people from other cultures. An individual 
IDI report provides an overall developmental score ranging from 55 to 145, along with scales and 
clusters specific to particular worldviews, as explained further in Table 1. 

Findings from Prior Studies of Education Abroad Programs 

An increasing number of education abroad programs have utilized the IDI as a program assessment 
measure. Between 2002 to 2008 the Georgetown Consortium study used the IDI to characterize 
intercultural learning among more than 1000 students in 61 US study-abroad programs, most of 
which did not actively facilitate intercultural learning. Males on average showed a decline and about 
one-third of females showed insignificant gains or declines. In addition, program design features 
traditionally believed to foster intercultural learning, such as home stays, were not always associ¬ 
ated with more intercultural learning and development (Vande Berg, Connor-Linton, & Paige, 2009). 

Some studies specific to engineering undergraduates show similarly small gains in intercultural 
sensitivity. Pre-post IDI data from global programs at John Brown University, University of California 
at San Diego, and Michigan Technological University (ranging from 4 weeks to semester-long) are 
reported in very different ways, but none found significant gains in intercultural sensitivity, and 
the vast majority of students did not progress into ethnorelative worldviews (Arzberger et al., 
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2010; Bland, 2010; Paterson, 2010). Furthermore, it was not uncommon for individuals or groups to 
seemingly regress on the overall IDI developmental score or on particular scales of the instrument 
(Arzberger et al., 2010; Bland, 2010). The first graduates of the comprehensive International Plan at 
Georgia Tech showed significant gains across a four-year period as measured by the IDI. However, 
students participating in their semester-abroad and short-term faculty-led programs showed small 
or negligible gains (Georgia Institute of Technology, 2011). Although these programs appropriately 
use multiple assessment measures, the quantitative IDI results suggest how difficult it can be to 
foster development of intercultural sensitivity. 

On the other hand, findings emerging from the education abroad research community indicate that 
significant progression on the IDI can be achieved when the intercultural experience includes proactive, 
intentional learning interventions (Paige & Vande Berg, 2012; Hammer, 2012). Of the variables examined 
in the Georgetown Consortium study, the one most associated with intercultural development was 
cultural mentoring, defined as "guided reflection on the students’ cultural experience.” Students who 
received the most individual and/or group mentoring showed significantly greater IDI gains (Paige 
& Vande Berg, 2012). These interventions are consistent with an experiential/constructivist learning 
paradigm, where students are guided to make meaning of their intercultural experiences. We aimed 
to incorporate this type of guidance in an education abroad program for engineering students. 


PROGRAM BACKGROUND 


Program Design 

As part of WPI’s project-based undergraduate curriculum, all students complete an interdisciplin¬ 
ary research project involving both social and technical dimensions, typically in their junior year. 
Conducted in small teams of students under faculty guidance, this project requirement is intended 
to help students learn how the social and cultural contexts of a problem impact its solution. Other 
learning outcomes are related to information literacy, teamwork and professionalism, and written and 
oral communication (WPI, 2015a). Engineering programs at WPI use these projects to develop and 
assess at least six of the eleven student outcomes in ABET’s Criterion 3 (ABET, 2014). Most projects 
involve addressing open-ended problems posed by community-based agencies and organizations. 
In the 2015-16 academic year, 518 WPI students (about 50% of the junior class) will complete this 
project requirement internationally at one of 21 Project Centers in Africa, the Americas, Asia-Pacific, 
and Europe (Davis & Mello, 2003; Mello, DiBiasio & Vaz, 2007; WPI, 2015b; Vaz, 2000; Vaz, 2005a). 

This study focuses on WPI’s Bangkok Project Center, which has been in operation since 1989 (Vaz, 
2005b). Each year, a cohort of about 24 students and two faculty advisors spend two months in Thailand, 
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working full-time on projects hosted by local non-profit, NGO, and governmental sponsors. The project 
topics typically involve issues related to sustainability, such as energy, environmental protection, public 
health, education, or community development. Recent projects have included development of tsunami 
mitigation plans, design of playgrounds for slum communities, analysis of environmental risk com¬ 
munication in vulnerable communities, and evaluation of land management techniques in rural areas. 

Pre-departure preparation has always been a core element of WPI’s Bangkok Project Center. 
Students receive 28 hours of Thai language and culture lessons from a native Thai instructor. To 
encourage students to give time and effort to language learning, this activity counts as 25% of their 
grade for the overall preparation experience, which also includes a course on research methods that 
guides them through development of a project proposal. In the language lessons the instructor also 
touches upon elements of Thai culture such as “do’s and don’ts” related to Buddhism, reverence of 
the Thai king and monarchy, respect for hierarchy and elders, and the value assigned to saving face 
and avoiding confrontation. In addition, two hours of pre-departure orientation meetings include 
discussion of culture shock and cultural adjustment. 

In Thailand, the WPI students and faculty are based on the Bangkok campus of Chulalongkorn 
University, although some projects involve extended fieldwork in rural areas. Students are housed 
in an international student residence hall. Cultural adjustment and cultural norms are revisited dur¬ 
ing orientation meetings shortly after arrival. Once projects are underway, all teams interact on a 
daily basis with Thai people at their sponsoring organizations and in communities impacted by their 
projects, in addition to others such as taxi drivers and vendors. In addition, students and faculty 
advisors discuss the cultural context of their work in an ad hoc manner. 

One might assume that the level of immersion and cross-cultural interactions just described 
would result in intercultural learning, but several research findings challenge that assumption. First 
and foremost, apart from pre-departure and on-site orientations, the WPI faculty leaders had never 
systematically engaged all students in identification of, and reflections on, cultural differences while 
on site. While faculty would certainly look for opportunities and discuss any matters the students 
brought to their attention, no proactive measures were taken. Another limitation of the WPI program 
model is its relatively short eight week sojourn. The Georgetown Consortium study concluded that 
program duration is significantly associated with intercultural gains abroad, with an optimal range 
of 13-18 weeks (Vande Berg, 2009). Knowing these limitations, the faculty leaders decided to take a 
more proactive approach for the 2009 program. 

Intervention Design and Rationale 

The design of this intervention incorporated cross-national teaming for some students and inten¬ 
tional intercultural learning activities for all students. During the months of January and February 
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2009, four of the six projects completed at the Bangkok Project Center involved mixed teams of 
WPI students and Thai students from Chulalongkorn University (CU). The CU students were enrolled 
in their fourth year of an English language International Program in Applied Chemistry, which has a 
requirement called the Science and Social Project modeled after WPI’s project requirement. There 
were two types of project teams: 

Mixed teams— three or four WPI students and two CU students, advised by faculty from 

both institutions; and 

WPI-only teams— three or four WPI students advised by WPI faculty. 

The mixed teams of students worked together on a daily basis for the eight-week period, co¬ 
presenting and co-writing along the way. Two of the four mixed teams worked in remote rural 
areas for four of the eight weeks, living together in addition to working together for that period. 
We anticipated that these WPI students would learn more about Thai culture from their Thai team 
members, but the Thai students were not charged with that function. 

The faculty augmented the pre-departure and on-site activities described in the previous sec¬ 
tion with additional instruction intended to foster intercultural learning more systematically and 
deliberately. So that students might develop more cultural self-awareness and be able to transfer 
their learning to cultures other than Thai, the additional learning activities focused especially on 
“culture-general” knowledge, skills, and attitudes that are useful for examining patterns in any culture, 
domestic or international (Bennett & Salonen, 2007; Bennett, 1998). In other words, there is a com¬ 
mon foundation upon which effective intercultural communication can be attempted, independent 
of the specifics of the new culture. 

Maximizing Study Abroad: A Program Professionals’ Guide to Strategies for Language and Cul¬ 
ture Learning and Use was utilized as the primary source of culture-general learning activities and 
facilitation strategies (Paige, Cohen, Kappler, Chi, & Lassegard, 2006). Students were also asked to 
purchase the student’s version of the same guide (Paige, Cohen, Kappler, Chi, & Lassegard, 2007). 
During the preparation period, students were assigned to read sections of Maximizing Study Abroad 
and to read a selection of essays by William Klausner (1993) illustrating certain aspects of Thai culture. 
They were then prompted to draft essays about two “core cultural value contrasts” from Maximizing 
Study Abroad for which they believed there would be noticeable differences between Thai cultural 
perspectives and their own. Examples of these value contrasts include individualism-collectivism, 
equality-hierarchy, meritocracy-ascription, and polychronic-monochronic views of time. The drafts 
were discussed with the students in a group setting, emphasizing ways in which appropriately cau¬ 
tious cultural generalizations can be explored without stereotyping. After this feedback, students 
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were asked to revise their essays. They were given a grading rubric explaining four criteria: critical 
thinking about culture; organization and coherence; writing mechanics; and effort to revise. The 
latter three criteria reflected the dual purpose of this assignment; written communication was also 
a core learning outcome of the project experience. Each student’s grade on this essay counted as 
10% of his or her grade for the preparation experience. 

Once in Thailand two activities were added to the usual orientation meetings. At the first meeting 
of the WPI and CU students, the faculty used an icebreaker called “What’s in a Name” suggested in 
intercultural training materials to create a sense of community (Lambach, 1996). Each participant 
writes his or her name on the board and tells the story of its origin. The faculty also administered 
a learning preferences inventory recommended as an intercultural training activity to trigger 
discussion within teams about both commonalities and differences (Hagberg & Leider, 1978). For 
example, individuals discuss their reliance on thoughts and feelings during decision-making, and 
preferences for action versus deliberation. After one week of project work, the advisors met with 
the WPI students to discuss types and stages of cultural adjustment they were experiencing, us¬ 
ing question prompts suggested in Maximizing Study Abroad. They also introduced the Describe- 
Interpret-Evaluate (D-l-E) process, which was perhaps the most substantive addition to the on-site 
cultural programming. 

The Describe-Interpret-Evaluate (D-l-E) process model is intended to help people separate their 
subjective reactions to a cross-cultural encounter from the objective aspects, and also to consider 
how a situation can be open to multiple interpretations that depend on cultural perspectives. In¬ 
tercultural communication experts recommend this disciplined sequence as a way to work through 
cultural frustrations and confusing incidents and to be more successful in interpreting behavior 
and events in unfamiliar cultures (Paige et al., 2007; Bennett & Bennett, 1991). Students were asked 
to choose an encounter they experienced that included “below the surface” elements related to 
culture, and then analyze it using the D-l-E model. As with the previous assignment, the students 
drafted essays and shared them with the advisors, who provided individual feedback. Students 
then revised and submitted their essays. The cultural learning revealed in this essay counted a small 
amount toward the project grade. 

In the last week of the sojourn, the WPI students met again with advisors for a meeting focusing 
specifically on preparing for re-entry and the likelihood of reverse culture shock. Again, question 
prompts for discussion were taken from Maximizing Study Abroad (pp. 143-146). 

The cultural learning activities just described are summarized in Table 2. All told, estimated stu¬ 
dent engagement in these activities was about 30 hours. These estimates do not include the time 
that students might have spent in informal discussions that took place related to their cross-cultural 
experiences. 
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Table 2. Summary of Intercultural Learning Intervention 


Cultural Learning Activities 

Estimated Student Involvement 

Pre-Departure 

Elements of Thai culture within the language course 

Thai culture and culture-general readings, essay, and revision: 
core cultural values and contrasts, stereotypes vs. generalizations 

Cultural elements of pre-departure orientation programs 

3 hours of instruction and discussion 

7 hours of reading, 5 hours of writing, 

1 hour of discussion 

1 hour of instruction and discussion 

On Site 

Cultural elements of on-site orientation programs 

Intercultural icebreaker and team-building exercises 

Describe-Interpret-Evaluate essay and revision 

Re-entry meeting 

4 hours of discussion 

2 hours of participation and discussion 

5 hours of writing, 1 hour of discussion 

1 hour of discussion 





METHODS 


Study Design 

The purpose of this study was to investigate the extent to which, and the means by which, the 
particular learning environment of the 2009 Bangkok Project Center facilitated intercultural learning. 
We aimed to address the following quantitative, qualitative, and mixed methods evaluation questions: 

1. To what extent did students’ intercultural sensitivity change over the course of the Bangkok Proj¬ 
ect Center experience? Were there differences between the U.S.-only and cross-national teams? 

2. How did students describe the cultural similarities and differences that they experienced? 

3. To what extent is the cultural learning evident from the qualitative results similar to or different 
from the quantitative results? 

A sequential explanatory mixed methods design was used for a single group of participants, col¬ 
lecting quantitative data first and then using qualitative data to add meaning (Creswell & Plano Clark, 
2011). The Intercultural Development Inventory (IDI) was administered before and after the education 
abroad experience as a quantitative measure of intercultural sensitivity. Although these quantitative IDI 
data certainly have benefits when assessing education abroad programs, students’ voices are absent. 
What do students in various stages of development sound like? How do they describe their intercultural 
experiences? Thus, a subset of the study population was interviewed upon their return, so that student 
voices about their intercultural learning could provide additional context beyond the quantitative data. 
The study protocol was submitted to the WPI Institutional Review Board, which granted an exemption. 

Study Participants 

Twenty-one WPI students participated in the Bangkok Project Center in 2009. All were juniors ma¬ 
joring in fields of engineering (20) or computer science (1), with the most common engineering majors 
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Table 3. Characteristics of Students in Potential Sample and Study Sample 



Number in 
Potential Sample 

Number in 
Study Sample 

Sex 

Female 

11 

10 

Male 

10 

7 

Citizenship 

us. 

18 

N.A. 

International* 

3 

N.A. 

Type of Project Team 

Mixed CU-WPI 

14 

10 

WPI-only 

7 

7 


*None of the students had Asian nationalities. 


being mechanical (6), chemical (5), and biomedical (4). A small monetary incentive was offered, and 
17 students gave their informed consent to participate in the study. The demographics of students in the 
potential sample and study sample, along with their types of project teams, are summarized in Table 3. 

Some incomplete information about study participants presents potential limitations. We cannot 
report the number of international students in the study group since gathering that information 
would have compromised the privacy of their decision whether or not to participate. In addition, 
information about students’ prior international or cross-cultural experiences was not collected. 
However, several previous studies indicate that prior study abroad or experience living in another 
culture are not necessarily associated with intercultural competence and do not predict IDI gains 
(Vande Berg et al., 2009; Thompson et a I., 2010). 

Several notes should also be made about team type (WPI-only or mixed). First, one of the teams 
designated as WPI-only did have interactions with and assistance from Thai peers at a rural univer¬ 
sity, but they were not assigned to work together and the Thai students did not receive academic 
credit. Second, although students were assigned to project teams, they were invited to rate their 
project preferences. This opportunity for self-selection according to team type may have resulted 
in differences between the students on the two types of teams. However, later results show no sig¬ 
nificant difference between the developmental IDI scores of students in the WPI-only and mixed 
teams at the start of the experience. 

Quantitative Data Collection and Analysis 

Intercultural sensitivity, as measured by the Intercultural Development Inventory (IDI), is the 
construct that was used as a quantitative indicator of intercultural development and learning 
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(Hammer et al., 2003). This measure was well-aligned with the program goal of advancing students’ 
experience and perception of cultural difference. In addition, the IDI is one of the most robust, 
highly regarded instruments in the area of intercultural learning, used in multiple education abroad 
programs both for diagnostic and evaluation purposes (Paige & Stallman, 2007). 

The IDI Version 2-3 (Intercultural Communication Institute, 2008) was deployed online to study 
participants by a qualified IDI administrator at the start of the preparation period (early September) and 
about one month after their return from Thailand (early April). Students were assigned an identification 
number so that their pre- and post-results could be linked while keeping their identity confidential. 
The results of the pre-test were not shared with participants, but students had the option of request¬ 
ing an explanation of their pre- and post-results after the conclusion of the study. The Developmental 
Model of Intercultural Sensitivity, which is sometimes used for instructional purposes, was purposely 
not introduced to the students to avoid influencing their responses on the post-test. Still, it is possible 
that students’ post-test responses may have been influenced to some extent by the pre-test. 

The IDI software returns a profile that includes several quantitative measures. A score from 1 to 

5 is provided for each of the instrument’s scales: Denial-Defense (DD), Reversal (R), Minimization 
(M), and Acceptance-Adaptation (AA). Drawing on a large data set of respondents, the software 
combines these scale scores into a standardized total (z-score), with a mean of 100 and a standard 
deviation of 15 (Hammer, Bennett, & Wiseman, 2003). This overall score ranges from 55 to 145. 
Subjects with overall scores ranging from 55 to 85 are categorized as having a dominant Denial/ 
Defense or Reversal orientation, from 85-115 as Minimization, and 115-145 as Acceptance/Adaptation. 
An overall score is reported for both “developmental” intercultural sensitivity—how the IDI rates the 
subject in developmental terms—and “perceived” intercultural sensitivity, which is how the subject 
rates himself or herself. Developmental scores are typically lower than perceived scores (Hammer 

6 Bennett, 1998). The perceived scores are useful when giving feedback to participants, but only 
the developmental score is meaningful for purposes of program evaluation. 

To explore whether participants’ intercultural sensitivity changed over the course of the experience, 
we used SPSS Statistics software (IBM Corporation, 2008) to conduct a two-tailed paired-samples 
t-test on matched pairs of pre- and post-IDI developmental scores. A two-tailed independent samples 
t-test was used to analyze whether the mean change in IDI score was different between students in 
mixed CU-WPI teams compared to those in WPI-only teams. Possible differences by gender were 
also explored; no significant effects were found. In light of concerns about the small sample sizes 
and normality assumptions of the t-test, analysis was also conducted using the equivalent non- 
parametric tests (Wilcoxon Signed Ranks and Mann-Whitney) and a permutations test (Good, 2000). 
Since the conclusions about statistical significance did not differ, we report the results of the t-tests 
due to their greater familiarity. 
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Qualitative Data Collection and Analysis 

Data from semi-structured interviews were used as a qualitative indicator of intercultural 
learning to bring students’ voices to the surface and to provide some context to the quantitative 
data. Interview questions addressed students’ experience of cultural similarities and differences 
and their sources of learning about culture. Core questions in the interview protocol included 
the following: 

• What commonalities and/or differences did you perceive between U.S. culture and Thai 
culture? 

• Can you give an example of a situation or incident in Thailand where you realized that cultural 
differences mattered or were at play? 

• Looking back now, what were the most useful sources of learning and insight into Thai 
culture? 

Six students were recruited for interviews using a stratified sampling method that attempted to 
have both sexes and team types represented in similar proportions as the whole sample. The same 
identification numbers were used so that interview results could be linked with IDI results. A pro¬ 
fessional staff member not involved in the study kept the code key of student names and study ID 
numbers. She began by contacting a randomly selected student from each of the six project teams, 
also aiming for an equal number of females and males. Students were offered a modest monetary 
incentive to be interviewed about the cultural aspect of their project experience. Ultimately, three 
females and three males were recruited, similar to the group as a whole. However, three interview¬ 
ees were from WPI-only project teams, which overrepresents that type of experience. The mean 
IDI developmental score of the six students who were interviewed was 86, compared to 90 for the 
whole study group. 

The interviews were conducted by phone by an independent professional interviewer, approxi¬ 
mately one month after the students’ return from Thailand, and shortly after administration of the IDI. 
The typical interview length was 30 minutes, and students’ responses were recorded and transcribed. 
Use of phone rather than face-to-face interviews was a logistical constraint of the interviewer; a 
clear limitation is the absence of non-verbal communication. In addition, findings from interviews 
are limited by their relatively short duration, which was believed to be necessary in order to gain 
the participation of busy students. 

One author with IDI training used NVivo8 software (QSR International, 2009) for open cod¬ 
ing of the data into themes (Patton, 2002). The qualitative portion of the study would have been 
strengthened by additional coders. Matrix coding queries were used to explore possible patterns in 
the types of cultural differences and commonalities identified by participants in different IDI stages. 
Quotations from study participants were used to communicate themes and patterns. 
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Table 4. Summary of Pre- and Post-IDI Results for Study Group as a Whole (N=17). 



Mean (Std. Dev.) 



Pre 

Post 

Change 

p-value* 

Overall developmental intercultural sensitivity + 

85.0 

(17.3) 

90.3 

(15.8) 

+5.28 

(13.5) 

0.126 

Denial-defense scale # 

4.18 

(0.53) 

4.16 

(0.49) 

-0.022 

(0.39) 

0.821 

Reversal scale 

3.40 

(0.89) 

3.58 

(0.60) 

+0.171 

(0.69) 

0.320 

Minimization scale 

2.31 

(0.61) 

2.31 

(0.72) 

-0.001 

(0.75) 

0.995 

Acceptance-adaptation scale 

3.12 

(0.57) 

3.91 

(0.33) 

+0.790 

(0.48) 

0.000 


+ The overall score may range from 55 to 145. 

'Two-tailed paired samples t-test. 

# Each scale ranges from 1 to 5, with a larger number always indicating developmental 
advancement through that particular orientation toward cultural difference. 


RESULTS 


Changes in Intercultural Sensitivity and the Effect of Team Type 

Students as a group showed modest positive development in intercultural sensitivity as measured 
by the Intercultural Development Inventory (IDI). Results of the pre- and post-tests are summarized in 
Table 4. The mean and median changes in their overall developmental IDI scores were +5.3 and +5.7, 
respectively. However, this gain was not statistically significant. Gain on the Acceptance-Adaptation 
scale was significant (p<.001). 

Most students remained in ethnocentric stages of development. Table 5 shows the number of 
students with each type of orientation at the beginning of the preparation process, and their ori¬ 
entations after returning from Bangkok. Fifteen of the 17 study participants were in ethnocentric 
stages of development at the beginning of the preparation process: eleven in Denial/Defense or 


Table 5. Changes in Worldview Orientation as Measured by the IDI. 



Number of Students 



Pre-Sojourn Orientation 

Post-Sojourn Orientation 

DDR M AA 

Denial/Defense or Reversal (DDR) 

11 

7 

4 


Minimization (M) 

4 


3 

1 

Acceptance/Adaptation (AA) 

2 


1 

1 

Total 

17 

7 

8 

2 
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Table 6. Summary of Mean Pre- and Post-IDI Results by Team Type: Students in Mixed 
Teams (n=10) versus Students in WPI-Only Teams (n=7). 




Pre 

Post 

Change 

p-value* 

Overall developmental intercultural sensitivity 

Mixed teams 

82.0 

92.2 

+10.2 

0.073 


WPI-only teams 

89.3 

87.6 

-1.70 


Denial-defense scale 

Mixed teams 

4.17 

4.20 

+0.031 

0.522 


WPI-only teams 

4.20 

4.10 

-0.097 


Reversal scale 

Mixed teams 

3.14 

3.60 

+0.456 

0.035 


WPI-only teams 

3.78 

3.54 

-0.237 


Minimization scale 

Mixed teams 

2.29 

2.38 

+0.087 

0.578 


WPI-only teams 

2.35 

2.22 

-0.127 


Acceptance-adaptation scale 

Mixed teams 

3.29 

4.02 

+0.869 

0.593 


WPI-only teams 

2.88 

3.75 

+0.735 



*Two-tailed independent samples t-test. 


Reversal (DDR) and four in Minimization (M). Reversal is a particular form of Defense that tends 
to romanticize other cultures. Examination of the Denial-Defense and Reversal scales shows that 
the group had already resolved issues of denial and defense before the preparation process be¬ 
gan, but that they were “in transition” regarding the Reversal orientation, meaning that they were 
still working through the tendency to view cultural differences from that mindset. This particular 
education abroad experience was successful in moving four students from DDR into the Minimi¬ 
zation orientation, but seven remained in the DDR mindset. Of the four students who started the 
preparation process with a Minimization orientation, one advanced to Acceptance-Adaptation 
while the other three retained a predominantly Minimization mindset. One of the initial two 
students with an Acceptance-Adaptation orientation remained there, while one returned with a 
Minimization orientation. 

A comparison of pre- and- post IDI scores by team type is shown in Table 6. Study participants 
in mixed teams showed a mean gain of +10.2 on the overall IDI developmental score, while students 
in WPI-only teams showed a mean decrease of -1.7. However, this difference was not statistically 
significant. (The marginal level of significance, p < 0.10, disappears if the pre-IDI score is considered 
as a co-variate.) Investigation of the scales reveals that students in the mixed project teams showed 
more progress resolving the Reversal orientation: +0.46 compared to -0.24 for the WPI-only teams 
(p < 0.05), meaning that they were less likely to believe that other cultures were superior to their 
own. There were no differences in other scales. 
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Figure 1. IDI developmental scores for each study participant, before and after the 
Bangkok Project Center experience, separated by team type. 


Changes in intercultural sensitivity for each study participant are shown visually in Figure 1, and 
the frequencies of various magnitudes of change are illustrated in Figure 2. Both figures illustrate 
that it was much more common for study participants in mixed project teams to make gains in IDI 
developmental scores. These figures also indicate that four students regressed in their intercultural 
development as measured by the IDI. Three of these students were from WPI-only teams. Exami¬ 
nation of the underlying scales shows that upon return these students tended to agree with more 
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Figure 2. Number of students showing various magnitudes of change in their overall 
developmental IDI score (post-pre), for students in WPI-only teams and students in mixed 
teams. 
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items that indicated Reversal and Minimization mindsets than they had before the preparation began. 
Both before and after the experience they had largely developed beyond a mindset characterized 
by Denial and Defense. 

Students’ Voices about Their Intercultural Learning 

Table 7 presents themes that emerged from qualitative analysis of post-sojourn interviews with 
six students regarding their experience of cultural differences and similarities. In order to provide a 
connection with the quantitative results, the team type, the post-sojourn IDI stage, and the change 
in overall IDI developmental score of each respondent are provided. 


Table 7. Themes in Qualitative Analysis of Six Student Interviews. Each interviewee is 
indicated by ID#, type of team (mixed or WPI-only), post-sojourn IDI orientation, and 
overall change in IDI developmental score. 


Theme 

Examples 

#18 

WPI 

DD/R 

-11.3 

#14 

Mixed 

DD/R 

-6.7 

#7 

WPI 

DD/R 

+7.0 

#20 

Mixed 

DD/R 

+8.0 

#4 

WPI 

M 

+1.3 

#11 

Mixed 

AA 

+1.2 

Cultural Differences 

Response to conflict 

Discomfort with criticism 

Indirect vs. direct communication 
Saving face 

✓ 



7 

✓ 

✓ 

Work behaviors 

Frequent breaks 

Integrating fun 

Less initiative 


7 


7 

7 

7 

Societal structures 

Respect for monarchy 

Influence of Buddhism 

Respect for elders 

✓ 

7 


7 


7 

Time-related behaviors 

Timeliness 

Keeping commitments 

Willingness to help 

✓ 


7 


7 


Aspects of daily life 

Restaurant service 

Frequency of eating 



7 


7 

7 

Social norms 

Discussing appearance 

Extent of gossiping 

Attitudes about partying 


7 




7 

Cultural Similarities 

Aspects of daily life 

Business world 

Pop culture 

Urban environment 

✓ 

7 

7 

7 



Individual needs 

Acceptance 

Building relationships 

Goals as college students 


7 

7 


7 


Universality 

Broad statements of commonality 


7 
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All students who were interviewed described meaningful cultural differences. Five of the six 
students spoke about conflict avoidance. The following quotations illustrating this theme are taken 
from interviews with two students whose post-IDI results indicated orientations of Denial/Defense 
or Reversal, and from one student with an Acceptance/Adaptation mindset: 

/ have a friend who was born in Thailand, lived there for about 10 or 12 years, came over 
to the States and because of money had to go and attend college in Thailand. / actually 
met up with him while I was over there. He is coming back to America because he can't - 
he doesn’t like the difference in the culture and what he specified and actually what / 
observed as well is that Thai people don’t generally deal with conflict. So if something 
goes wrong, if you are late to a meeting, it never really gets brought up. It just gets kind 
of pushed and swept under the rug, which is I think very different from America. I mean 
I’m from the Northeast and / think the Northeast maybe is a little even more extreme 
to the rest of America sometimes in regards to this. But usually generally you're very 
upfront with people especially in a work situation. You were late, why were you late, this is 
unacceptable. Whereas in Thailand it was kind of like oh whatever, we won’t even ask them 
about it. (#18, DD/R) 

As Americans often times we see everyone as special and important and they sort of 
see the whole as important—they don’t want to stand out from one another as much 
as i think we’re encouraged to do as Americans. So I think they were hesitant to bring 
up any points of contention or conflict, possible conflict. And my other two American 
group mates a few times got into arguments about how we should, if we should focus on 
preparing for meetings and having group discussions or if we should focus on writing. 

They got noticeably upset with one another. And the two Thai students just sat there and 
stared at their papers and didn't say a word. / tried to like mediate the situation and...they 
didn’t do anything. They didn't do anything at all. / tried to kind of appease both of them. 

(#20, DD/R) 

We were asked to do this thing called a teamwork assessment where we pointed out each 
team member and how they contributed and how they can improve....So we tried to do it 
every week. And sometimes it would be impossible because you would have to tell someone 
their weakness and why you think they should improve on this. And the Thai students didn’t 
feel comfortable doing that at all. The one girl in my group wouldn't even open her mouth; 
she wouldn't even participate because for them it's so rude. (#11, AA) 
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The first two students completed the sojourn with the same IDI orientation of DDR and described 
a similar cultural difference, but in somewhat different ways. Student #18, whose developmental 
IDI score decreased, implies that he and his Thai friend believe that one cultural norm is better 
than the other and does not offer an explanation of a Thai perspective. In contrast, Student #20, 
whose developmental IDI score increased, identifies an underlying difference in perspective— “the 
importance of the whole” versus the importance of the individual—to explain the behavior she 
observed. The student with an AA orientation also explained the cultural difference in terms of the 
Thai partner’s perspective. 

Another common theme that emerged in interviews was differences in work behaviors. Two stu¬ 
dents, one with a Denial/Defense or Reversal orientation and another with Minimization, discussed 
the importance their Thai partners placed on breaks, fun, and being happy: 

We found especially that when we were doing our own work for the report we would only 
be able to work in maybe half an hour to an hour increments and then we would take a 
break because the Thai students were accustomed to that. / mean we had no problem with 
taking breaks.... / think around the world people are not as you know not—hard working 
is not the word I’m looking for, but like you know obsessed with work as Americans are. 

And that comes out, the way they take breaks, and it's just kind of a fun, you know relaxed 
atmosphere. The word in Thai is sanook for fun and it’s a very, very important part of 
everything. (#14, DD/R) 

Another difference is when we finally got to work the Thai students seemed much more— it 
was more important that we were happy than we actually got work done. So if we went to go 
do something- they'd take us out to eat first if they thought we were hungry or they would 
want to please us to make sure everyone was happy before getting down to the hard work. So 
that’s a lot of difference. [In the U.S.] it would be that if you were to get a group of students 
or anybody for a project that / feel like the host would want to get to work also as opposed to 
make sure that everybody’s happy, because this is the working environment. (#4, M) 

The different IDI stages of those two students are not obvious in the excerpts, and the student 
with a DD/R mindset describes how his team adjusted their behavior—taking more breaks—to be 
more consistent with a Thai workplace. Note that this student’s developmental IDI score decreased 
from pre- to post-sojourn. 

Within the same theme of work behaviors, two other students, one with a DD/R orientation and 
the other with an AA orientation, described hesitancy to take initiative as a cultural difference. They 
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described their observations in similar ways, despite their contrasting ethnocentric and ethnorela- 
tive worldviews as measured by the IDI: 

Our Thai partners were really hesitant and my other American classmates noticed the same 
thing with the Thai students in their groups. They were really hesitant to speak at meetings and 
really had to be prompted. / mean they were not used to professor/student interaction that 
way. They were used to just being lectured at and not really speaking at all during class and 
giving their opinions or ideas. They had like a really hard time with that. I’m not sure it could be 
the language. Her English wasn’t bad, it was quite good. But / think it’s also just that they're not 
- they just seem to not take initiative that way like to go up and start a conversation. They sort 
of wait for somebody to come to them for instance. / don't know how to word it. (#20, DD/R) 

Working style is very different too. They were uncomfortable doing a lot of things, being 
spontaneous. They were mostly used to being told what to do in a way. I guess creativity is 
not as promoted as strongly over there as it is over here. / guess it’s also because English is 
not their first language so of course they wouldn’t be as confident as a native speaker. But 
/ felt like a lot of times they were too afraid to step in and take the initiative. But once you 
told them you should do this section they would get it done very fast. (#11, AA) 

While all students who were interviewed identified multiple cultural differences, it was more difficult 
for them to identify cultural similarities. Four students, all of whom were characterized by the IDI as 
having a Defense/Denial or Reversal mindset (see Table 7), identified surface features of daily life such 
as the necessity for eating; urban activity; the presence of American movies, music, and TV; and similar 
dress and use of cell phones in the business world. The following example is representative of this theme: 

/ think maybe the professional aspects of Thailand and America and probably everywhere 
are very similar in the fact that there’s business people and they’re on their cell phones 
and they're dressed up and they’re doing their own thing. I don’t know if I expected that 
necessarily. (#7, DD/R) 

One student in the Defense-Reversal mindset also showed indications of working through no¬ 
tions of universality: 

/ mean you know to a certain extent everyone is the same right?... And you know there’s 
certain things that are universal around the world. (#14, DD/R) 
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Thus, in the interview this student was showing some indications of a Minimization mindset- 
developmental progress— even though his dominant orientation was DD/R and his developmental IDI 
score declined. One student in a mixed project team was not able to identify any cultural similarities, 
responding "That's hard. / don’t know.” This student (#11) was in the Acceptance-Adaptation stage 
of development both before and after the education abroad experience. 

Sources of Cultural Learning 

One of the more important and useful findings of this study is the stronger IDI gains of students 
who were immersed in cross-cultural, cross-national project teams, spending many hours together 
working toward a common goal in an interdependent manner. Two of the three interviewees who were 
in these mixed project teams identified their Thai team members as important sources of cultural 
learning. The following excerpts suggest how and what they learned from their Thai partners: 

/ fee/ like it was a very useful and important part of the experience for me to be in a team 
with the Thai students. Though the main way I learned from them was just by talking with 
them about cultural differences and things that I noticed and asking, you know, where does 
this fit in the Thai culture? (#14) 

The girl in my group was about my age and she was from a very conservative family. But at 
the same time when we were together it didn’t feet like she was at all. It felt really natural. 

But then when I saw her around Thai people and around her family or something like that / 
could tell that the way she behaved was very, very different from the way I behaved when I’m 
at home. So that helped me see a lot of the culture because I considered her like "a typical 
Thai girl”...I really liked the things / learned from her because she had so much respect for her 
professors and older people. And she would always serve everyone food before she served 
herself. And it's very - like small details that were very different from my culture. (#11) 

Five of the six students who were interviewed also described their Thai language teacher as an 
important source of learning about culture during the preparation period. 


DISCUSSION 


Efficacy of the Intervention 

Despite the fact that participants in this study did not show statistically significant gains in inter¬ 
cultural sensitivity as measured by the IDI, the mean change in the IDI developmental score for the 
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Table 8. Compilation of IDI Results from Education Abroad Programs for STEM 
Undergraduates and Other Undergraduates. 


Authors and Program 


Length of 
immersion 


Mean change 

in IDI 

Sample developmental 
size score 


Mean pre- and % advancing 
post- IDI scales* at least one 
(S-point scale) IDI stage 

R M AA 


STEM Undergraduates 


This study 


Whole study group 

8 weeks 

17 

5.30 

3.40- 2.31- 3.12- 
3.58 2.31 3.91 

29% 

Students in cross-national teams 

8 weeks 

10 

10.20 

3.14- 2.29- 3.29- 
3.60 2.38 4.02 

30% 

Georgia Tech (2011) 






International Plan graduates 

Study abroad 

At least 6 months 

Semester 

29 

40 

6.90 

2.71 

Not reported 

47% 

43% 

Faculty-led programs 

Varied, 4—16 weeks 

117 

0.42 


34% 

No abroad experience 

- 

462 

-0.89 


28% 

Arzberger et al. (2010) 

9 weeks 

21 

Not reported 

3.57- 2.49- 3.12- 
3.17 2.42 3.44 

Not reported 

Paterson (2010) 

10 weeks 

6 

1.6 

Not reported 

Not reported 

Bland (2010) 

4-6 weeks 
Semester 

64 

16 

Not reported 

Not reported 

22% 

31% 

Non-STEM or Mixed Undergraduates** 

Anderson et al. (2006) 

4 weeks 

23 

4.22 

3.72- 3.08- 3.05- 
4.11 2.90 3.42 

13% 

Georgetown Consortium (2009) 

Varied 

1163 

2.37 

Not reported 

Not reported 

Pedersen (2010) 






With intercultural pedagogy 

One year 

16 

11.56 

Not reported 

Not reported 

No intercultural pedagogy 

One year 

16 

1.22 




*R= Reversal; M= Minimization; AA= Acceptance/Adaptation 
**See Vande Berg, Paige, & Lou (2012) for additional studies. 


study group as a whole is among the highest reported for STEM undergraduates participating in 
education abroad. Table 8 shows a compilation of study results. Authors have reported IDI data in a 
variety of ways that makes complete comparisons difficult. Assessing the available data, however, 
we can say that the mean developmental IDI gains observed in this study are the highest we could 
find for a short-term (less than a semester) education abroad program for STEM undergraduates. 
Furthermore, the gains exceeded those for some semester-long programs. 

Small sample size and large variance are certainly contributing factors to the absence of a statis¬ 
tically significant change in the IDI developmental score for the group as a whole. A small number 
of students showed relatively large decreases in their IDI scores. These decreases may be due to 
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the fact that the instructional activities were not tailored for each student’s developmental stage. 
For example, some intercultural trainers recommend quite different goals and activities for learners 
with a predominant Defense/Reversal mindset than for a learner with a Minimization orientation 
(Bennett & Bennett, n.d.). It is quite possible that some of the students in mixed project teams, par¬ 
ticularly those who started at a lower developmental stage, were overwhelmed by the challenging 
experience, did not have sufficient support, and retreated as a consequence. Others maintain that 
intercultural development may proceed along non-linear pathways (Bourjolly et at, 2005), such 
that short-term decreases in intercultural sensitivity do not preclude a longer-term advancement 
emerging from the same experience. 

Both the quantitative and qualitative data suggest that Thai people may have been the most 
influential source of cultural learning for students participating in the Bangkok Project Center. 
The quantitative data indicate that students in the mixed project teams showed significantly more 
progress than students in WPI-only teams in working through the Reversal form of Defense—the 
tendency to romanticize other cultures. (One team categorized as WPI-only relied significantly on 
Thai students who were not officially part of the team. Had this team been categorized as mixed, 
the difference in IDI gains between students in mixed and WPI-only teams might have been even 
larger.) In interviews, all three of the students in mixed project teams spoke of teamwork challenges 
and frustrations that they attributed to cultural differences. Closer interaction with Thai people may 
have been a contributing factor in helping these students resolve their tendency toward Reversal. 

The design of this study does not enable us to judge the incremental benefit of the program 
design and cultural learning intervention compared to the smaller-scale curriculum used in the past. 
In addition, the study evaluated the effects of the experience as a whole; we cannot credit the gains 
to the cultural learning intervention. In fact, as a group, students in the WPI-only project teams did 
not show advancement in intercultural sensitivity as measured by the IDI, despite their participation 
in the same learning activities. Still, it is possible that without the cultural learning curriculum, stu¬ 
dents would not have had the perspectives and tools to help them process confusing or frustrating 
incidents and encounters. Indeed, five of the six students who were interviewed spoke positively 
about the Describe-Interpret-Evaluate (D-l-E) assignment for processing cross-cultural encounters. 
The following two examples are representative: 

It [the D-l-E assignment] helped look at a situation more objectively instead of just jumping 
right to a conclusion. (#20) 

Writing about it and thinking about it in the D-l-E way—process— definitely kind of gave me 
a new idea to think of other conflicts that I come upon when there’s a little misunderstanding 
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somewhere instead of immediately putting my opinion on it and just to try and look at it from 

their point of view and interpret it a little more before explaining it. (#4) 

More discussion of students’ perceptions of particular elements of the cultural learning curriculum 
can be found in a previous publication (Demetry & Vaz, 2010). 

Implications for Program Design and Interventions for Intercultural Learning 

Because of the quantitative and qualitative results showing the value of mixed project teams, the 
decision was made to use all mixed teams of students at the Bangkok Project Center in subsequent 
years. In addition, other WPI project centers in China, India, Namibia, and South Africa have incor¬ 
porated local college students or residents into project work as co-researchers. After the first year 
of experience in Bangkok we made adjustments to strengthen cross-cultural exchange among the 
project teams. The teams are now more balanced, with four WPI students and three CU students. 
(Perfectly balanced teams would be preferred, but programmatic constraints prevent that.) Further¬ 
more, upon arrival of the U.S. group in Bangkok, several days focus on fun and socializing among 
team members. In addition, WPI and CU faculty members are working more closely together when 
advising the mixed project teams. 

Much more needs to be done to better support both CU and WPI students in mixed project teams. 
In interviews, the WPI students on those teams described frustrations and cultural differences with 
little evidence that they were able to articulate or empathize with Thai conceptions of effective 
teams, or to navigate and manage the differences with growing confidence. The cultural learning 
activities utilized in this intervention were limited, in large part because of all of the other work that 
students were being asked to do. Students would no doubt benefit from more extensive and skilled 
“cultural mentoring” which education abroad experts define as guided reflection on cultural experi¬ 
ence (Vande Berg et al., 2009). Successful models for cultural mentoring can be found in several 
non-engineering programs. Pedersen (2010), for example, describes an extensive program of cultural 
mentoring in which U.S. undergraduates spending one year in England were given individual coach¬ 
ing tailored to their IDI profiles, a curriculum on group dynamics from social psychology, community 
building activities, and guided reflection assignments. As shown in Table 8, this approach yielded 
significant IDI gains, albeit in a year-long program. Engle and Engle (2012) describe a Cultural Pat¬ 
terns class that provides cultural mentoring for students participating in the American University 
Center of Provence. In the Georgetown Consortium Study, those students gained, on average, 12.5 
points on the IDI developmental scale, significantly outperforming students in 60 other programs 
(Vande Berg et al., 2009). Others have described greater use of the Maximizing Study Abroad: Stu¬ 
dents' Guide , asking students to make regular e-journal entries to reflect on their use of intercultural 
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learning strategies (Cohen, Paige, Shively, Emert, & Hoff, 2005). A key element in implementing 
these pedagogical approaches among STEM educators would no doubt be more extensive training 
on intercultural development and cultural mentoring (Paige & Goode, 2009). 

Entirely different approaches for developing students’ intercultural sensitivity are also being ex¬ 
plored at WPI. Many of the resources that we used are in the tradition of Geert Hofstede’s research 
on cultural contrasts— bipolar dimensions such as Individualism/ Collectivism (Hofstede & Hofstede, 
2005). While these contrasts may be a useful point of entry for college students, others have called 
it an “essentialist” perspective— the idea that a set of attributes can identify a group of people- 
criticizing it as “sophisticated stereotyping” that assumes cultural homogeneity within nations (Jones, 
2007; Osland, Bird, Delano, & Jacob, 2000). Because of this concern, two other WPI project centers 
are experimenting with the notion of “otherness” in an attempt to enhance students’ awareness and 
understanding of culture. During the pre-departure phase, students are asked to reflect on questions 
of who, how and why certain individuals in American culture are designated as being different or 
not belonging to their group. Toward the end of their international project experience they again 
share their sense of otherness, this time focusing on the culture of the host country (Elmes, Hersh, 
& Shockey, 2012). This approach has been successful in other education abroad contexts (Roose, 
2001) and may be more effective in increasing students’ cultural self-awareness, which is essential 
for development according to the Developmental Model of Intercultural Sensitivity. 

Revisiting the Assessment of Intercultural Competence 

This study also aimed to examine the connections, if any, between the quantitative IDI results and 
how student interviewees reflected on cultural differences they experienced. For engineering edu¬ 
cators at least, some of the interview results can be difficult to reconcile with outcomes suggested 
by the IDI. In interviews, even students with ethnocentric worldviews, according to the IDI, revealed 
many cultural insights and the ability to interpret some events and behaviors from alternative cultural 
perspectives in a non-evaluative way. For example, one student with a DD/R orientation who had 
shown regression on the IDI—presumably a negative outcome—showed the ability to analyze cultural 
differences about views of euthanasia in a reasonably sophisticated and non-judgmental manner. 

A larger study of study abroad students in Minnesota showed differences between IDI results 
and qualitative indicators of learning (Cohen et al., 2005). While students in an experimental group 
engaged in strategies-based instruction for intercultural learning showed no greater gains on the IDI 
and other quantitative measures than students in a control group, two qualitative measures identi¬ 
fied specific ways in which the intervention had made a difference in student learning. 

Limitations in the qualitative portion of this study prevent us from taking full advantage of the 
mixed methods design, and we cannot evaluate whether the quantitative and qualitative results are 
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consistent or not. Hammer (2012, p. 127) laments the “quantitative versus qualitative war” in the study 
abroad field, asserting that when developmental interviewing is used, there is a strong relationship 
between the two types of data. In a developmental interview, students are asked to provide accounts 
of specific situations or critical incidents that they experienced, explain the cultural differences at 
play and strategies they used to navigate those differences, and describe their perception of the 
outcomes of the incident (Hammer, 2012). This study would have benefited from an interview protocol 
that included all of those elements, and from an interviewer with knowledge of the Developmental 
Model of Intercultural Sensitivity, a larger number of interviews, and multiple coders. 

Regardless of whether quantitative and qualitative results are consistent, many stakeholders and 
decision-makers who review IDI program assessment data may not be told of or understand its nu¬ 
ances, and may conclude from small or negative changes in IDI scores that programs do not result 
in learning gains. For example, one participant in this study who showed overall regression in the IDI 
developmental score indicated progress working through Reversal, the tendency of romanticizing 
other cultures. Another showed advancement on the Minimization scale while maintaining a primary 
DD/R orientation. Depending on how the quantitative results are reported, these areas of progress 
may not be evident. As mentioned previously, some scholars point to qualitative evidence that 
intercultural development may proceed along non-linear pathways, such that transient regression 
can be a segment of longer-term growth (Bourjolly et al., 2005). 

Many engineering education stakeholders also may not understand the very high standard set 
by the IDI. A study by Goode (2007) showed that a group of faculty members leading education 
abroad programs at a private liberal arts institution were characterized as “in transition” on both the 
Reversal and Minimization scales of the IDI, not unlike most of the undergraduate students in this 
study. Anecdotal evidence from intercultural trainers suggests that at least 40 hours of targeted 
training may help individuals advance one stage on the IDI (Hammer & Bennett, 1998). Furthermore, 
people with an Adaptation mindset, capable of sophisticated cognitive and behavioral frame-shifting, 
typically have deep understanding of, and experience living and working in, at least two cultures. 
Thus, pre- and post-program administration of the IDI may not be the appropriate tool to judge the 
quality of education abroad programs or to characterize student learning outcomes. 

Indeed, a recent study showed that among a group of higher education administrators, 90% 
agreed on the value of pre- and post-tests to assess intercultural competence, compared to only 
65% of a group of internationally-known intercultural scholars (Deardorff, 2006). Quantitative pre¬ 
post tests may be alluring to engineering administrators and faculty due to ease of administration 
and analysis and because of a preference for numerically-measured outcomes. The IDI is even more 
alluring because of its robust psychometrics. Intercultural experts, on the other hand, tend to believe 
in the development of intercultural competence as an ongoing process, one that should be monitored 
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and measured over a longer time period, not just at two points in time, and not associated with just 
one intervention (Deardorff, 2006). 

For all of those reasons, combined with the logistical constraints of required certification and 
cost of administering the IDI, the Bangkok Project Center Directors at WPI have not continued using 
the IDI as a pre-post program assessment measure for intercultural competence. The IDI is perhaps 
most powerful as a diagnostic measure; results can be used to guide instruction and can be shared 
and discussed with student groups. Some alternative measures are being used or developed in the 
engineering education community. We believe that scenario-based assessments (Downey et al., 2006; 
Jesiek & Woo, 2011) hold particular promise as direct and authentic assessments of intercultural or 
global competency. 


CONCLUSION 


Overall, this research reinforces both the challenges and opportunities to develop and assess 
students' intercultural competence in engineering education abroad programs. While the sample 
size is quite small and the results cannot be generalized, combined quantitative and qualitative 
evidence in this study points toward the value of extensive interaction between students in cross¬ 
national project teams, accompanied by intentional instruction and reflection on cultural differences. 
Individuals in teams consisting only of students from a U.S institution showed much less progress. 
More extensive and skillful cultural mentoring for students in mixed teams is the next step in im¬ 
proving the program model. 

Another outcome of this study is a cautionary note about use of the Intercultural Development 
Inventory (IDI) as a quantitative program assessment and student learning outcomes measure, at 
least as the sole measure, despite its undeniable quality as an instrument. The qualitative portion 
of this study enabled us to identify some positive learning outcomes that nonetheless did not 
translate into the significantly high standard required for advancement on the IDI. Since education 
administrators tend to value pre-post tests (Deardorff, 2006), minimal gains or losses on the IDI 
may put sound programs at risk. Use of the IDI to measure change in intercultural sensitivity over 
the entire college experience, from matriculation to graduation, as is being done at Georgia Tech 
(2011), seems to be a more appropriate use of the instrument. 

Finally, this study reinforces the arguments of many engineering education scholars about the 
challenges associated with defining and assessing intercultural or global competence. What do 
varying degrees of intercultural competence look like for engineering undergraduates? What de¬ 
velopment goals are realistic to achieve, and how can change be measured in an authentic manner? 
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Work in this direction is ongoing (Downey et al., 2006; Jesiek & Woo, 2011) and may help engineering 
educators design, assess, and improve education abroad programs and other curricular elements 
that will better prepare students for global engineering practice. 
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